Liver tumors are the fifth most common tumors and the second leading cause of death in cancer. Focal Liver Lesions (FLLs) are abnormal solid or cystic masses in the liver. The discovery of FLLs in the early stage of liver cancer and the diagnosis of FLLs may have important significance for the treatment of liver cancer. In the process of diagnosis, medical images have played a very important role, and especially in recent years, with the development of an imaging technology, the location of medical images in diagnosis is more and more important. Medical images comprise Computerized Tomography (CT), Magnetic Resonance Imaging (MRI), and Ultrasound (US) imaging, among which the CT and the MRI require high costs and complex instruments, and the CT may also cause ionizing radiation. The US imaging has become more and more widely used because of its low price, fast imaging, and noninvasive imaging. However, due to an imaging mechanism of ultrasound, images obtained by the US imaging are blurred, have a low resolution, and have a low signal-to-noise ratio. The recently proposed Contrast-Enhanced Ultrasound (CEUS) imaging method studies a dynamic enhancement pattern of FLLs over a period of time by continuously imaging FLLs over a period of time. By studying the differences and changes of an FLL region with respect to surrounding healthy tissues over time, CEUS can significantly improve the detection results of FLLs. The use of CEUS images for assisted diagnosis not only helps doctors to obtain more information and improves the efficiency of diagnosis, but also reduces unnecessary pain for patients.
During the actual diagnosis of liver tumors using CEUS images, a doctor usually injects a contrast agent into the blood vessels of a patient. As the contrast agent flows along with blood in the body, the CEUS images will form four main phases namely a plain phase, an arterial phase, a portal phase, and a delayed phase. The duration and imaging features of each region are different. A radiologist identifies the FLLs, typically by observing an enhanced change form of a lesion region over the three phases (arterial phase, portal phase, and delayed phase). The portal and delayed phases are mainly used to distinguish between malignant cancers, such as Hepatocellular Carcinoma (HCC), and benign cancers, such as Hemangiomas (HEM) and Focal Nodular Hyperplasia (FNH). Most of the malignant cancers show a low enhancement in portal and delayed phases while most of the benign cancers show flat enhancement or high enhancement. On the other hand, the arterial phase may provide useful information for distinguishing specific liver cancer categories. For example, the vast majority of HCC cases are highly enhanced in the arterial phase, while some cases are not uniformly enhanced or show a circular enhancement around a larger nodule. In benign cases, most of HEM cases show a peripheral nodular increase, another part shows a rapid and uniform high enhancement. FNH cases may show a spoke-like enhancement or a uniform high enhancement in the arterial phase. Different changes in these same cases described above will also be considered at the time of diagnosis.
At this stage, the accuracy of diagnosis depends heavily on the experience and level of a diagnosis doctor. At the time of diagnosis, doctors often need to repeatedly view the entire section of CEUS image, find the location of the lesions in the images and an imaging mode of the lesions, and finally diagnose the cases according to their own medical knowledge or medical knowledge in literatures. The diverse and complex enhancement pattern of the liver cancer mentioned above also brings great difficulties in distinguishing different FLL types. In addition, the CEUS imaging is ambiguous and requires experienced doctors to perform detailed observations to identify and diagnose. This usually takes a lot of time to deal with every case. At the same time, the length of a section of CEUS image is usually 3 to 5 minutes. When the number and data of patients increase, it will undoubtedly require a lot of labor and time for the doctor.
On the other hand, the field of computer vision has developed rapidly in recent years, and many good results have been achieved in the field of natural images and imaging such as object recognition, positioning, detection, scene classification, segmentation, video tracking, motion recognition and even semantic analysis. With the rapid development of the field of machine learning, the combination of computer vision and machine learning is becoming more and more intimate. Computer vision has begun to not only simply process images and video itself, but has begun to understand and process the content and even semantics of images and videos. On some data sets, the accuracy of the method for detecting objects has even exceeded the human itself. The data and structure of medical images in computers are not essentially different from natural images. Information about a pixel is represented by a single value. Therefore, methods and technologies in computer vision may be transformed into the field of medical images. The method of analyzing image and video content in computer vision may also be practiced and applied in the field of medical images. If the analysis and understanding of a medical image can be realized, the computer may be used to assist a doctor in aided diagnosis. By giving the key part and the key time in the image, the imaging and changing features in an image will be given, which may save plenty of time for the doctor's diagnosis. For new physicians, Computer-Aided Diagnosis (CAD) systems can help or guide them in identifying lesions, which is an important aid to training experienced doctors. So in reality, the analysis of medical images has a very considerable application prospect and value.
As mentioned above, with the development of computer technologies such as a medical image processing technology, machine learning, pattern recognition and a computer vision technology, CAD systems have also been developed and applied. In many other medical fields such as breast cancer, CAD systems have achieved good results by assisting physicians in the analysis and interpretation of medical images. At present, the relatively mature technologies of CAD systems for medical images comprise de-noising, segmentation, registration, 3D reconstruction, and so on. De-noising comprises: preprocessing an image, such as adjusting a contrast and sharpening the image, so that imaging is more conducive to the doctor's observation; the segmentation is to separate the same organ or region in an image or image sequence from other parts, and prepare for the next step of the CAD system; the registration is to match the same part of different types of medical images in the same case, thereby making it easier for doctors to view the same region in different medical images; imaging of some techniques such as CT is to scan a part of a human body layer by layer to form a two-dimensional tomographic image; and the 3D reconstruction is to combine these two-dimensional images to form a three-dimensional model of an organ or region.
Although a current CAD system has a certain degree of application in these aspects, most of them are dealing with medical images. The existing systems mainly focus on segmentation of tissues and organs, interactive and automatic segmentation of focal lesions, edge detection and so on. The medical image processing is also mostly concentrated on images such as CT that are clearly imaged and easy to handle. In the FLL recognition and diagnosis part of a CEUS image, because of the disturbance of a lesion region in three imaging stages of the CEUS image and various imaging patterns, few CAD systems can analyze and recognize FLLs in the CEUS image. Even some of the cutting-edge methods rely on manually determining the locations and regions of the FLLs. The accuracy of manual annotation is highly related to the doctor's technical and domain knowledge. Different doctors also have different understandings of the lesions, which may cause the annotation time and location to be slightly different. On the other hand, with the increase in acquisition and processing technologies, the number of CEUS data is growing at a fairly high rate. Manual annotation requires a lot of time for doctors. Therefore, a fully automated CAD system for analyzing and diagnosing FLLs is extremely necessary.
In the field of medical image processing, there is not much work to recognize liver FLLs in CEUS images. Some methods use a quadratic curve to fit the average grayscale temporal changes in the lesion region to represent an imaging pattern of lesions in CEUS imaging to distinguish the lesion type and separate FNH from other lesion types; or the lesion region is manually segmented, and multiple cascaded neural networks are used to classify lesions. There is also a method to propose a Dynamic Vascular Pattern (DVP) to represent the imaging features of the lesions. During the test, the surrounding normal tissue and lesion regions are manually marked in a certain frame, an average grayscale curve of two parts in the entire image can be automatically generated, and then this curve is used to distinguish between benign and malignant tumors, thereby achieving a very good effect. In the above work, more or less interactions are needed to determine the location of lesions or normal tissues manually. Human interaction relies heavily on the knowledge, skills and experience of an operator, and it is easy to make different doctors have different interpretations of the same case, thereby causing disturbances in results. On the other hand, with the continuous growth of ultrasound image data, if doctors need to manually interact with each case, they will consume a lot of energy and time, and manual annotation of all data will become more and more difficult. Therefore, a system that can automatically perform aided diagnosis without manual interaction is necessary.
In another field of medical image processing, it focuses on automatic detection or segmentation of other types of tumors. By using grayscale information, these methods can find edge and region features, and ultimately achieves segmentation of multiple tumors in a variety of medical images.
Another type of treatment for liver tumors in ultrasound images is to track liver tumors in consecutive CEUS images. Due to human respiration and motion, jitter of an operator and the like, the tumors in the ultrasound images will often change locations and sizes, and sometimes they will be blocked, causing the grayscale of the tumors to change or even disappear. Most of the methods in this field perform tracking by region matching via various features or slight disturbances. Or, a spatial relationship between local information of a region and the region is considered at the same time, and a model is used to jointly express the two. To prevent tracking errors from accumulating over time, some methods also use models to express a relationship between image appearance information and tumor bias. It starts with human respiration and tries to correct the offset locations of the tumors by performing template matching on the tissues in some frames, or by finding out the errors caused by human respiration, and finally, the tracking accuracy is improved. Determining the location of the lesion is a very important step in the recognition of this method, but these methods still do not recognize the FLLs. The field of medicine still does not directly recognize the types of lesions.